Fundamental Tensor : On the Geometry of ThreePerspective

Reconstruction of 3D space from 2D images is generally a \2-view" problem. That is, the algebra and geometry of the problem is all contained in measurements across two views, the rest (i.e., when more than two views are available) is statistics. This paper embarks on the investigation of the algebra and geometry of three views. Based on the recent result 18] showing the existence of certain trilinear functions of three views with a corresponding tensor of 27 intrinsic coeecients, we derive a set of relations between the tensor and geometric invariants of the 3D world, intrinsic structures of two views (fundamental matrix and epipoles), and show the existence of new intrinsic structures (beside the tensor itself) associated with three views. On the practical side, the tensor ooers a host of new algorithms for recovering 3D information from 2D views. These methods cut through the epipolar geometry (i.e., three views are used simultaneously, rather than in pairs, reconstruction proceeds independently of epipolar geometry), make room for statistics (additional views can be incorporated to obtain further redundant equations, just like in traditional 2-view-based schemes), and generally exploit the information available from measurements across views in a more eecient manner than any technique based on 2-view geometry.